Digital Darkfield Decompositions 
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Lattice imaging facilitates digital implementation of optical darkfield strategies that have for 
decades played a key role in the conventional electron microscopy of materials. Applications to 
the microscopy of periodic structures are described in the context of recent developments in the 
mathematical harmonic analysis community, with hopes of inspiring further development along 
these lines. Applications here, which push present day limits using sharp-edged Fourier windows 
alone, include the location of weak periodicities in images by virtue of their spatial correlations, and 
the quantitative mapping of projected strain in a variety of nanostructures. 



PACS numbers: 03.30.+p,01.40.Gm,01.55.+b 

I. INTRODUCTION 

The word wavelet is less familiar in microscopy than 
in other areas of appUed math. However, microscopists 
have long been versed at interpreting images which trade 
spatial resolution for information on transverse momen- 
tum (e.g. on scattering direction, for example). As a re- 
sult, microscopists (particularly those involved in trans- 
mission electron microscopy of crystals) have been de- 
veloping skills connected to the use of combined space 
and frequency decompositions since well before the coin- 
ing of the word wavelet in the 1980'e^. This is partly 
because microscopists do their decompositions with help 
from optical (rather than digital) computation, wherein 
their analysis hides behind the more familiar appellation: 
darkfield imaging (cf. Hirsch ct al.^). 

Here we discuss a few ways microscopists, and other 
periodicity analysts as well, might bring evolving digital 
tools of harmonic analysis to bear on problems tradi- 
tionally addressed in analog fashion on live images using 
sharp-edged back focal-plane apertures and a variety of 
time-honored optical darkfield techniques. In the pro- 
cess we seek to bring their challenges to the attention 
of the oft-times application-inspired harmonic analysis 
community'^''*. 



II. THE DARKFIELD IMAGE TRANSFORM 

The elementary space plus frequency decomposition 
in signal processing is sometimes viewed optically as a 
windowed Fourier transform, in which the window and 
exponential kernel of the original transform combine to 
become the new kernel of the harmonic transform^. In 
this context, such "windowlet" transforms in ID might 
be written as 
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where s[t] is the input function, t is a direct-space (or 
time) shift parameter, fo is a frequency shift parameter. 



and A/ is a frequency-space scale parameter (recipro- 
cal to the position/time-scale parameter At). Note that 
we are using dimensioned parameters here, even though 
the kernel function h[£^,rj\ is a function of dimensionless 
quantities. If the kernel can be written as the product 
of a window function w[^] and a modulation function 
— e~'^^^^'^ , then the Fourier transform of the win- 
dowlet becomes simply 



^Sj[f,fo,Af]^S[f]W 



f-fo 
A/ 



where S[f] and W[ri] are the Fourier transforms of s[t] 
and w[^], respectively. 

When the Fourier window is the simplest type of 
optical window to manufacture, namely a sharp-edged 
aperture of width A/ centered at frequency fo, then 
W[r]] = Rect[ri] = If[|77|<i, 1, 0]. Hence the direct space 

window is w[^] — ''"^^^^^ = sinc[^]. 

When |/o|<-^, microscopists refer to the map of in- 
tensity ip*'ip as a brightfield image (exclusively so when 
illumination is parallel) because it includes the unscat- 
tered beam. Regions in the image without scatterers are 
therefore bright. Conversely, when the image does not 
include the unscattered beam microscopists call the tp*'tp 
map a darkfield image. The corresponding phase maps, 
by comparison, may contain very precise information on 
the lattice strain tensor in projection. This transform 
lends itself to rapid calculation of a series of fast Fourier 
darkfield decompositions that bridge the gap in a variety 
of ways between direct and reciprocal versions of a given 
dataset. Issues related to integrability, admissibility, and 
regularity are considered in more detail below. 

Because this sine (or Shannon/Littlewood-Paley) win- 
dowlet is the one with which darkfield microscopists have 
the most optical experience, it's uses will be the focus of 
this paper. We expect, for work on atomic and lattice 
resolution digital images, that applications familiar to 
microscopists optically will benefit from the much more 
diverse set of transforms possible digitially. One purpose 
of this paper is therefore to bring these applications to 
the attention of the signal processing community. 

In particular, strategies for reciprocal space "tiling" so 
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FIG. 2: Left: (a) Traditional wavelet tiling of reciprocal space; 
Right: (b) Wavelet tiling rearranged into "diffraction pattern" 
format, with a centered DC peak. 



FIG. 1: Amplitude images of a 2D spatially-delimited periodic 
array (e.g. a small crystal in projection). Inset a) is the 
starting brightfield dataset; b) is an image formed from a 
conjugate symmetric pair of diffracted beams and the DC 
peak; c) is the same as inset b without the DC peak - note the 
doubled periodicity because of the doubled spacing between 
interfering tiles; d) is the amplitude of a complex darkfield 
formed using a rather small single tile; e) is the same as inset 
d using a larger tile in frequency space; f) is the same as 
inset d using an even larger tile in frequency space - note the 
uncertainty principle in action in this last series of three. 



the application^, this is a practical handicap for exam- 
ple when locating nanocrystal edges in otherwise noisy 
images. Improved spatial convergence is hence one area 
where digital signal processing (for example by refining 
a decomposition to recognize lattices of discrete bumps 
in the 0.2 nanometer size range) might contribute down- 
stream. Even the utility of existing decompositions with 
good frequency resolution (e.g. brushlets^ and ridgelets^) 
for these applications remains to be examined. 

In the 2D case, using cartesian vectors and a common 
bandwidth A/ for each direction, the windowlet trans- 
form becomes 



as to sample all of frequency space are an area of ac- 
tive interest in the signal processing community today 
(cf. Flesia et aim-). However, the focus is often largely on 
compression techniques. A very different set of consider- 
ations is at work in the microscopy community, particu- 
larly in the study of high resolution transmission electron 
microscope images of periodic lattices, and defects or in- 
terfaces associated therewith. With recent developments 
in atomic-resolution digital microscopy allowing acquisi- 
tion of near-gigabyte images by a variety of techniques, 
the opportunities for signal processing detective work on 
the atomic scale are quite rich. 

Before we move on to the 2D transform, we should 
ask whether this choice of w[£] meets the usual qual- 
ifications for wavelet decomposition. Although it is a 
direct-reciprocal reversal of the "short-time Fourier win- 
dow" decomposition oft mentioned in the literature on 
wavelets, this sinc-function window has zero mean and is 
square integrable. It's transform also meets the admissi- 
bility condition (at least in the parallel illumination case) 
provided |/o| > i.e. when the frequency window does 
not include the zero-frequency point or "DC peak" . This 
of course is consistent with the standard definition of 
"darkfield image" in microscopy, i.e. an image formed 
from waves which do not include the unscattered beam 
so that regions free of scatterers look dark. The weak 
point of the decomposition, of course, is that it exhibits 
sinc-system convergence cx i as f — > oo, at one end of 
the Harr to Sine system continuum^^. Although the num- 
ber of vanishing elements required depends heavily on 
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If the kernel h can further be written as the product 
of a window function w [^] and a modulation function 
.9[^i'?] = e~'^^^^*^, the Fourier transform of the win- 
dowlet becomes 



vE'[f,fo,A/] -5[f]iy 



f fo 

A/ 



Finally, a square Fourier window (for simplicity) looks 
like W[r]x,r]y] = = Kect[r]x] Rectfry^]. This results in 
a 2D direct-space sine- window of the form 'w[^xiS,y] — 
sm[^g3,] smf^jy] ^ rpj_^g shape of the Fourier window is of 
course arbitrary mathematically. For optical applica- 
tions, particularly in the electron microscopy case when 
the physical aperture size is in the 10 to 100 micron size 
range, circular rather than square apertures are more 
common (and easier to manufacture as well). 



III. NON-TILED CIRCULAR, GAUSSIAN, AND 
BAYESIAN BACKGROUND-SUBTRACTED 
DARKFIELDS 

One feature of the sine windowlets defined above, 
which may at first glance be unfamiliar to microscopists, 
is that they are generally complex. Fourier filtering of 
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FIG. 3: Linear decomposition heirachies with 4- fold symme- 
try: (a) 4 X 4, (b) 8 X 8, (c) 16 x 16. 




initially real images traditionally preserves their conju- 
gate symmetry in reciprocal space, via selection of re- 
gions (e.g. to window) that are symmetric about the DC 
peakiS. The complex darkfield images described here do 
not. As a result, however, they reclaim something that is 
normally lost in Fourier filtering of images: The continu- 
ous (fringe-free) illumination of scattering centers (e.g. of 
crystals in the field diffracting electrons into the selected 
Fourier aperture). This is because darkfield imaging with 
windows which may be asymmetric about the DC peak, 
of the sort described here, relegates periodicity fringes 
in the images largely into phase, rather than amplitude, 
variations in the image. By simply examining amplitude 
(or amplitude squared) across the images, these varia- 
tions disappear just as they do in the microscope when 
image intensities (and not phases) are recorded in the 
imaging process. These advantages of asymmetric dark- 
field are illustrated in Fig. ^ 

As long as tiling is not a requirement of the analysis 
process, for example when one simply wants to see where 
in the image the source of a peak in the power spectrum 
is located, any imaginable window function (not includ- 
ing the DC peak) may be used to form a darkfield image. 
Given the growing ability of microscopes to provide large 
digital images containing abundant crystal lattice fringe 
information, one can thus use these wavelets in the com- 
puter to do darkfield imaging just as in a transmission 
electron microscope, with the added advantage that one 
can be much more creative, and less limited by the phys- 
ical challenges of aperture design, than is possible in the 
microscope. 

For example, microscope apertures usually have cir- 
cular symmetry (at least by design). In the case of 
a circular opening in a microscope's back-focal plane, 
W[r], (j)] = Rect[ri\ in polar coordinates, and the direct- 
space window function becomes the well-known Airy dis- 
tribution — "^^ expressed in terms of the 
Bessel function of the first kind J„. Although sharp- 
edged apertures are often easiest to manufacture (at least 
for electron optic applications), convergence in mathe- 
matical applications can be improved by the use of soft- 
edges (e.g. the Gaussian Fourier window functions as- 
sociated with the Morlet wavelet). Creative choice of 
the window function can do more than improve regular- 
ity. For example, such functions can be "tuned" with 
Weiner filtering for the Bayesian removal of broad spec- 



FIG. 4: Decomposition series of n x n square Fourier window 
decompositions running from a real direct space image in (a) 
with n = 1 to the Fourier transform in (h) with n = 128. Hue 
denotes the Fourier phase of each pixel, with red and cyan 
corresponding to the positive and negative reals, respectively. 



trum background in imagesi^^, or for the recognition 
of patterns of interest in images, such as edges. Digi- 
tal darkfield analysis, in that sense, provides yet another 
context for their continuing refinement. 

In addition to use of different "back-focal apertures" 
of the form VF[r/], this strategy also facilitates a number 
of "decoherence" strategies, some of which (like hollow- 
cone darkfield) have been implemented optically, and 
others which have not. This is done by forming sepa- 
rate darkfield images in more than one chosen (typically 
local) range of frequencies, and then either combining 
or cross-correlating the intensities (square amplitudes) 
found therein. Thus annular darkfield images might be 
formed by adding intensities in patches forming one or 
more annular rings in the power spectrum, thus target- 
ing regions responsible for a given "powder diffraction" 
signature. Alternatively, one might cross-correlate inten- 
sities found in patches associated with a specific pair of 
reflections, thus searching for cross-fringes from a partic- 
ular single crystal lattice orientation in a large image. 



IV. TILING STRATEGIES AND 
DECOMPOSITION HEIRARCHIES 

On the subject of decomposition heirarchies (or 
Daubechies' "frames" formed using a discrete sublattice), 
consider the attached image files in sequence. Figure 12^ 
illustrates the traditional array of horizontal (01), ver- 
tical (10) and diagonal (11) components generated as a 
2D image is broken down into wavelets (Axx) along with 
a rescaled (reduced resolution, or "brightfield" ) image in 
the upper left corner, which of course may be as shown it- 
self decomposed (Bxx, etc.) at each stage with less direct 
space but more reciprocal space resolution. One might 
characterize this as a series of 2 x 2 decompositions. 

Figure ??b simply rearranges that 2x2 heirarchy ac- 
cording to the "locations" in frequency space about which 
each frame is centered, with the DC peak in the field lo- 
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cated in the center. Note that the first stage (A) compo- 
nents correspond to higher frequencies, and also higher 
bandwidths (more resolution) than do the higher stage 
(e.g. B, C) components - a feature that distinguishes 
wavelets from short time Fourier decompositions, for ex- 
ample, and makes them better for wide dynamic range 
phenomena and "situations where better time-resolution 
at high frequencies than at low frequencies is desirable"—. 
Note also, however, that the angular resolution of fre- 
quency (subtended at the DC peak) at any given decom- 
position stage is very poor. 

FigureOJi does the same "recentering" , except this time 
for a 4 X 4 decomposition heirarchy. Here, the angular res- 
olution is improved, and the "wraparound" edge effects 
(most noticable at the corners) affect fewer elements. Be- 
cause frequency as well as spatial resolution are crucial 
in our work, if we have n x n images where rt>16, then 
the optimum tiling for us is not 2 x 2 or 4 x 4, but some- 
thing like y/n x y/n. Figures |3Jd andO; sketch the 8x8 
and 16 x 16 decompositions in this series. Remarkably, 
in a way which may be largely independent of the basis 
functions involved, when this decomposition is taken to 
it's n X n limit on an n x n image, one gets simply the 
Fourier transform itself as illustrated in Fig01 

None of the foregoing discusses the functions used for 
the decompositions, but only their locations and domains 
in reciprocal (and scale) space. The simplest conceiv- 
able functions to "center and scale" to these domains 
are sharp edged window functions. This is especially 
true when constructing electron optical filters: You make 
for example a 10 micron hole in a thin piece of metal, 
and electrons are either transmitted with unit amplitude 
and no phase shift where the hole is, or they are not 
transmitted at all. Partial transparency, or phase mod- 
ulation, would require controlled aperture thicknesses at 
the atomic scale, and hence have not historically been 
a practical choice. The "darkfield perspective" on po- 
sition/frequency localization is of course not limited to 
such "all or none" aperture functions, but they are a 
natural place to start because of their convenience and 
history of use in microscopy. 

The natural path between direct image and Fourier 
transform, that this way of looking at hierarchies pro- 
vides, is more than just theoretical. Figure |5l shows what 
happens when one forms an image of the wave field in 
an electron microscope just above or below the objective 
back focal plane of a periodic specimen (in this case sap- 
phire) under plane parallel illumination. (The Fourier 
transform itself resides in the back focal plane.) As you 
can see, the decomposition of the previous figures can (it 
would seem) be found grinning back at us from the midst 
of the electron wavefield in our scope! Note that here the 
circular pattern of each inset maps the effective shape of 
the specimen region being illuminated. Here as in later 
examples, symmetries for the direct space source region, 
and the Fourier tiling, may be chosen independently (cf. 
Figs. ©. 

Also note that the symmetry of the "tiling" is not cho- 




FIG. 5: Defocussed electron diffraction pattern of a crystal, 
showing that sinc-decompositions intermediate between direct 
and frequency space can be found in nature. 




FIG. 6: Logarithmic Fourier window tilings with (a) four- 
fold symmetry in both the initial (direct space) image as well 
as in the Fourier windows used, (b) polar symmetry Fourier 
windows applied to an initial circular image source region, 
and (c) alternating equi-triangular Fourier windows applied 
to an initial hexagonal image source region. 




FIG. 7: Reciprocal space domains associated with a diffrac- 
tion spot array from the 2D reciprocal lattice of an arbitrary 
crystal lattice, viewed in projection with small Bragg angles 
down a low-symmetry zone axis. 
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sen with apertures, but by the specimen itself. This sug- 
gests to me that the wavefield in the microscope may be 
more generahy modeled as a kind of image/FT convolu- 
tion (is Wigner-Ville transformation relevant here?). It 
masquerades as a Fourier-space tiling here, because this 
specimen's "point-rich" 3D reciprocal lattice acts (via it's 
intersection with the incident electron Ewald sphere) as 
a 2D reciprocal comb (i.e. as a periodic array of delta 
functions in frequency space). 

The "microscopist's perspective" also provides some 
clues to the types of more sophisticated Weyl-Heisenberg, 
Wigner-Ville and/or wavelet-like decompositions (e.g. 
Meyer- Coifman brushlets^) that might prove useful in 
helping microscopists make the most of their digital im- 
ages (especially those that show atomic lattice resolu- 
tion). In this latter case, the fact that the spacing be- 
tween all atoms in solids is on the order of 0.2 nm means 
that precise information on both location and scale size 
may be desired with a very specific frequency range in 
mind. The structures we search for (discrete arrays of 
equally spaced molecules) are also very specific. It turns 
out in this context, for example, that simple rules define a 
continuum of tilings between square and hexagonal, that 
have been optimized (as in Fig. 01 to offer maximum 
expression of the shape transforms associated with each 
spot in an arbitrary 2D lattice. Of course, this strategy 
is only useful if you know the periodicity and orientation 
of the lattice of interest. In the absence of this infor- 
mation, or in the presence of more than one crystal, the 
symmetry of the tiling scheme may be dictated by other 
considerations (or the lack thereof). 

Also, specific periodicities and their harmonics is a spe- 
cialized interest. If on the other hand adaptivity to a wide 
range of scale sizes takes precedence (especially relevant 
for images greater than a gigabyte in size), logarithmic 
tilings as in FigO which extend the wavelet strategy of 
treating frequency as a scale parameter, may be of help. 
Many ridgelet tilings, for example, adopt this tact. 

In FigureElD, the dark circles represent (as do the discs 
in Figure O the shape in direct space of the region be- 
ing analyzed, while the light blue patches in which they 
are centered represent the frequency domains from which 
these "darkfield image patches" draw power. Likewise 
for the dark hexagons in Figure One of the nice 
things about sharp window frequency space decomposi- 
tions (like those commonly used for electron microscope 
darkfield imaging) is that digital algorithms exist today 
(some quite efficient) for tiling schemes like those of Figs 

EE 



V. DIGITAL DARKFIELD APPLICATION 
EXAMPLES 

The examples here illustrate (i) the use of sinc- 
windowlet intensity maps to highlight the distribution 
of a selected periodicity in an image, (ii) the use of tiled 
decompositions to search for patterns of periodicity dis- 



tribution in space which might otherwise be "covered" 
by noise, and (iii) the use of sinc-windowlet phase maps 
to plot selected strain components associated with each 
of several lattice planes whose periodicities are projected 
into a single image. All begin with images taken using 
a 300kV Philips EM430ST transmission electron micro- 
scope (TEM) with point resolution near 0.2 nm. 



A. Visual tracking of thin-film nucleation and 
growth 

A look at CU2O electrochemically grown epitaxially 
onto the surface of a (100) silicon wafer by Jay Switzer 
(cf. Switzer et. alM^) at the University of Missouri in 
RoUa (UMR) illustrates the usefulness of sinc-windowlet 
analysis, both in identifying barely visible periodicities, 
and in using digital darkfield images to highlight period- 
icity distributions. The specimen was prepared in cross- 
section by cleaving along (110), followed by preparation 
of a 3 mm diameter "stack" for slicing, abrasion, dim- 
pling, and Argon ion milling. 

An excerpt of a larger high resolution TEM electron 
phase contrast (HREM) image, covering a region just to 
the left of the primary Cu2 0/Si interface, is shown in 
Figure |S1 top left. The right side of this image shows 
parts of two octahedral Si pyramids with (111) faceted 
walls. The larger image shows that these pyramids jut 
upward from the (100) silicon surface, pointing in the 
direction of CU2O epitaxy. The left side of the image, by 
comparison, contains lattice fringes from the CU2O layer 
itself. Of particular interest is the question of growth 
nucleation. These two phases have a large lattice misfit, 
and the CU2O itself shows evidence of low angle grain 
boundaries in the epitaxy, suggesting that a mechanism 
for accomodating misfit through disorder is at work. 

Unfortunately, the presence of amorphous material 
(e.g. silicon oxide) as well as a 50 nm layer on top and 
bottom of the specimen due to ion milling are conspir- 
ing to make the lattice fringes fairly difficult to see or 
track throughout the image. To get a closer look, there- 
fore, we calculated a 16 x 16 darkfield decomposition of 
this image (Fig|Sltop right) along with a regular Fourier 
power spectrum (Fig |H1 bottom right). From the power 
spectrum, of course, interspot spacings and angles allows 
us to compare fringe spacings and angles rather quan- 
titatively. The darkfield decomposition, by comparison, 
allows us to tell where in the image the "power" or vari- 
ance, in a given patch of frequency space, is concentrated. 

The eight blue circles in the top right of Figure |S| high- 
light regions of frequency space whose variations are con- 
centrated in the right half of the image, i.e. in the silicon 
pyramids. One can tell this because in these Fourier win- 
dows the right half of the patch is noticably brighter than 
the left half. The four orange circles highlight regions of 
frequency space whose variations are concentrated in the 
left half of the image, i.e. in the CU2O lattice. 

This is confirmed by a look at these corresponding re- 



6 




FIG. 8: Top left; 512 x 512 excerpt from 15000 x 15000 image 
of a Cu2 0/Si interface (running from around 7 o'clock to 1 
o'clock just to the right of the region shown in the image) 
in cross-section. This excerpt shows parts of two pyramidal 
Si mounds on the right, and some fringes on the left from a 
cuprous oxide column that likely nucleated between them (fol- 
lowed by columnar growth orthogonal to the substrate silicon 
wafer, viewed here down (110)); Right side: maps calculated 
from Fig|H| On top is a 16 x 16 darkfield decomposition show- 
ing spatial correlations (left CU2O, right Si) within selected 
frequency windows. At bottom is a power spectrum for mea- 
suring periodicity spacings and angles; Bottom left: Color 
overlay from a larger partion of the HREM image discussed 
here, rotated so that the Cu20/Si interface is horizontal. The 
orginal HREM image is in green. A darkfield version of the 
same image, formed using only intensity from Si lattice peri- 
odicities, is in blue. In red, find a darkfield image formed us- 
ing only intensity from the particular 4-degree rotated CU2O 
lattice discovered in the other figures 



gions of frequency space in the power spectrum in Fig 
IHl (bottom right) . Measurement of the spacings and an- 
gles confirms that the blue-labeled spots correspond to 
Si spacings viewed down the (Oil) zone, with the (400) 
reflection perpendicular to the Cu2 0/Si interface in pro- 
jection. The orange spots correspond to a CU2O (Oil) 
zone, with the (200) reflection about 4 degrees clock- 
wise from the interface normal. The CU2O lattice is also 
twisted between 0.5 and 1 degree "up" about the beam 
normal, because the (020) spot is not visible while the 
(040) spot is there if extremely weak. In fact, this 1.07 
Angstrom (040) spot would not have been noticed or con- 
firmed without the evidence for spatial correlation with 
the CU2O side of the image in the corresponding fre- 
quency window of the darkfield decomposition at top. 
Given this spot, we are able to determine the orientation 




FIG. 9: Left: HREM image of a Pd icosahedral twin formed 
by solid state reduction in polyvinyl chloride; Right: Power 
spectrum of the above image showing elements of 10-fold sym- 
metry. 

of the CU2O column to better than a degree in any direc- 
tion. Future work with even more careful filtering might 
allow inspection of the epitaxy between CU2O and Si, for 
evidence of dislocations at points of Cu20/Si contact. 

Work on the larger image from which this was drawn 
confirmed that the "4 degree off" column of CU2O indeed 
extends up to the top of the layer. But where near the 
interface does it begin, and how can we map the location 
of this particular crystal "from a distance" , i.e. in a figure 
sufRciently reduced in size that the whole film can be seen 
even though the lattice fringes themselves are then too 
small to resolve? 

FigurelHlbottom left illustrates with a composite which 
began with the reduced image of a larger part of the orig- 
inal negative in green. Overlaid in blue find a darkfield 
image created by adding intensities from Fourier win- 
dows which pass primarily periodicities of the silicon sub- 
strate. As you can see, the silicon pyramids with thin 
silicon at the top show up especially well. Lastly, in red 
(which combines with green to form a coppery orange) 
find a darkfield image created by adding intensities from 
Fourier windows which pass primarily periodicities of the 
"4 degree off" CU2O column. The image confirms the 
columnar growth pattern, and moreover offers some rea- 
son to suggest that the epitaxy probably takes place near 
the peak of the Si pyramids, rather than at their base. 
This would offer opportunities for the CU2O lattice to 
attach near the tip of this pyramid (or along it's side), 
taking advantage of the high local curvature to select a 
minor tilt to help accomodate the lattice misfit. In this 
way, digital darkfield imaging might allow one to visual- 
ize processes on scales much larger than the periodicities 
being effected. We expect that this use will become more 
important as even larger images, with atom-scale metric 
information in them, become available. 

B. Single image recognition of icosahedral twins 

In this section, we treat a more complex problem, 
namely the study of twinned nanoparticles. Although the 
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FIG. 10; a: Digital darkfield image using a Pd (200) reflec- 
tion, showing the predicted "bowtie" twin pair; b: Digital 
darkfield image using a Pd (111) reflection, showing the pre- 
dicted "butterfly" twin quartet; c: 1024 x 1024 HREM image 
of a "randomly oriented" and twinned AuPd cluster; d: Dark- 
field 0.14 nm (220) butterfly, discovered in a 32 x 32 darkfield 
search of the image above for i-twin "bowties" and "butter- 
fiies". 

stable configuration for large crystals of a given metal at 
room temperature is often well known, when metal par- 
ticles are smaller than 10 nm in size, surface to volume 
ratios are much larger and modes of crystal growth not 
energetically favorable (or even impossible) in bulk are 
quite common. The classic example of this is icosahedral 
growth, perhaps facilitated by the low surface area of tri- 
angular fee (111) facets in an icosahedral arrayl^. Surface 
considerations begin to pale as such particles acquire vol- 
ume, and what began as an icosahedral assemblage grows 
into a set of 20 paired and nearly tetrahedral fee twins 
(one twin for each icosahedral face). 

The HREM image of a particle of icosahedrally 
twinned fee Palladium, formed by solid state reduction 
in polyvinyl chloride, microtomed, and viewed down the 
5-fold axis, is shown in the top of FiglHl Although sym- 
metry in the structure may not be apparent from the 
image, the central region of the power spectrum (bottom 
Fig ISJ reveals a clear 10-fold, or pseudo 10-fold, sym- 
metry. The inner- most spacings are consistent with 0.22 
nm fee Pd (111), the intermediate spots are consistent 
Pd (200) spacings of around 0.19 nm, and the occasional 
outer spot is consistent with the 0.14 nm Pd (220). Of 
course, fee crystals (in fact, all translationally periodic 
lattices in 3D) have no 5 or 10 fold orientations. 

Digital darkfield images, formed using the periodici- 
ties surrounding a single spot in the Fourier transform, 
provide a striking answer to the question: What's going 
on? For example, the top left of Figure HIH was obtained 
by windowing the region of frequency space marked (a) 
in Fig 1^1 As a result, the set of Pd (220) planes run- 



ning from lower left to upper right in one pair of twins 
in the particle "lights up" . Electron microscopists might 
see this as "forming the image with electron Bragg scat- 
tered into the Fourier aperture by those two pyramidal 
crystals" , although of course no diffraction is taking place 
in this analysis at all. Similarly, each set of (111) spac- 
ings in an ideal fee icosahedral twin is shared by four 
of these crystals. When viewed down a 5-fold direction, 
such (111) planes "light up" a butterfly pattern like that 
in the bottom left of Fig EH This image was obtained 
by windowing only those frequencies surround the spot 
marked (b) in the power spectrum of Fig ^ 

The images in Figure 1101 will be familiar to materi- 
als electron microscopists as darkfield images with "the 
g-vector of the active reflection" drawn in (cf. Hirsch et 
alii^. The small size of this twin particle, and the crowded 
nature of it's reciprocal lattice, would make this exper- 
iment difficult to perform optically, but the strategy is 
familiar. Consider now, however, the problem of identi- 
fying icosahedral twinning in a randomly-oriented nano- 
particle. Individual 2-second exposure HREM images 
can include lattice information on hundreds of nanoparti- 
cles, but can harmonic analysis allow one to detect icosa- 
hedral twinning after the fact? 

The top right of Figure ^| is a HREM image of a 
twinned AuPd cluster provided by Massimo Bertino at 
UMR (cf. Doudna et ali^). The specimen is on a car- 
bon support film. Its power spectrum (not shown) con- 
tains spot spacings characteristic of AuPd, but with ap- 
parently random orientation thus offering no evidence of 
symmetry in the specimen. However, a 32 x 32 tiled dark- 
field decomposition of this 1024 x 1024 image (analogous 
to that in FiglSj) contained at least one bowtie/butterfly 
structure like those in Fig EH The clearest of these (a 
bowtie) is shown in the bottom right of EH where the 
g-vector orientation associated therewith is the one ex- 
pected for icosahedral twinning as well. 

The numerical calculation of 1023 tiled darkfield im- 
ages (and one brightfield image) which made this discov- 
ery possible was done in essentially the time required for a 
single forward and reverse 1024 x 1024 fast Fourier trans- 
form. We are currently doing calculations to determine 
the fraction of such icosahedral twins likely to be inden- 
tifiable from a single, randomly oriented, image. Back 
of the envelope considerations, as well as the data shown 
here, suggest that the method has a signficant chance of 
success. 

Even if one could optically detect diffraction spots from 
this particle, and set up darkfields for each of the diffrac- 
tion spots in the pattern, the time involved would be pro- 
hibitive. Since this calculation can also be done as well 
for the hundred or more other particles represented in the 
quarter gigabyte of information from a single 2-second 
HREM image exposure, this strategy of tiled darkfield 
decomposition shows potential for offering both extended 
sensitivity, and major savings of time, in the study of 
nanoparticle assemblages. 
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FIG. 11: Complex color phase (left) and strain (right) maps 
for bowties (top) and butterflies (bottom) from a simulated 
icosahedral twin. 



C. Mapping lattice strain around 2D and 3D 
defects 

Thanks in part to the fact that transmission electron 
microscopes record data "in parallel" as distinct from 
"in series" (as is the case e.g. for scanning probe micro- 
scopes), HREM images under optimum conditions can 
map the projected potential in a thin specimen with rea- 
sonably high metric fidelity. Hence distortions measured 
in HREM lattice images can provide rather precise in- 
formation on lattice structure in a specimeni^. In this 
section, we discuss how the Fourier phase of darkfield im- 
ages can therefore provide details about specimen struc- 
ture e.g. on percent-level lattice strain components as 
projected into the beam direction. The extent to which 
the process of thinning the specimen, as well as strain re- 
laxation at the surface after thinning, modifies the state 
of the observed specimen from it's state e.g. prior to 
thinningiii^ will not be discussed. We will also here ig- 
nore effects, like local specimen thickness or instrument 
defocus, which might in themselves result in shifts in the 
image Fourier phase from point to point. Thus we will 
content ourselves with the mathematician's task of local- 
izing periodicity in the images, and leave to others the 
task of inferring what those periodicities might mean for 
the specimen itself. 

Let's begin with a theoretical example. The left half 
of Figure 1111 contains maps for bowtie (top) and but- 
terfly (bottom) darkfield images from a modeled icosa- 
hedral twin particle, rendered in "logarithmic complex 
color" . This means that brightness in the image is loga- 
rithmically related to the windowlet amplitude, while the 
phase is rendered using a cyclic hue-related color table 
(see inset) in which red (for example) might correspond 
to positive real and cyan negative real. A similar color 




FIG. 12: Log reciprocal-amplitude directional complex-color 
strain image for the GaN (003) "growth direction" periodic- 
ity which runs perpendicular to the GaN/sapphire interface 
(upper left corner). 

scheme was earlier used in Figure 0] to represent complex 
numbers. 

Fourier phase shifts in a periodicity amount to local 
displacements. For example, in one spatial dimension an 
interval with sinusoidal oscillations would be shifted by 
7r/2 from an adjacent interval with cosine-usoidal vari- 
ation. If, as in the cases discussed here the darkfield 
image is dominated by power in a single lattice period- 
icity, then the 2D gradient in that phase describes the 
2D strain one-form, the components of which are sim- 
ply change in lattice position per unit traverse in a given 
direction. The log magnitude of this strain one-form is 
inversely proportional to brightness in the images on the 
right side of Fig ^2 while the direction of that strain 
is denoted by hue (as per the key in the center of the 
figure) . 

Note that the strain seems random in areas with win- 
dowlet amplitudes down in the noise (i.e. away from the 
object with strong periodicities in this frequency win- 
dow). In these areas, strain fluctuations occur on a scale 
size inversely proportional to the bandwidth of the dark- 
field window. In regions of high amplitude the strain by 
comparison seems relatively uniform, i.e. on the bow- 
tie and butterfly patterns themselves. However, note the 
blue bands centered on the butterfly wings in the lower 
right quadrant of Fig ^] This is the strain associated 
with the tensional (outer) half of the partly compres- 
sional, partly tensional (111) interface between adjacent 
"bowties" (paired crystal 3-sided pyramids). It arises 
in essence because the fee geometry is asking for a 70.5° 
(^ arccos [1/3]) angle between similar sections, while the 
icoshedral symmetry demands 72° (<— 27r/5). Note that 
the direction (hence hue) of the interface strain shows 
it to be directed perpendicular to the interface between 
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crystals (as well as the planes associated with the win- 
dowed periodicity). The short range of the strain is a 
result of the detailed icosahedral model used: No "relax- 
ation length" was included in the calculation. 

This theoretical example in hand, consider now an ex- 
perimental example. Figure 1121 shows the strain image 
obtained as described above, for the (003) growth di- 
rection periodicity at the bottom of a "GaN/InGaN on 
sapphire" molecular beam epitaxy quantum well speci- 
men (cf. Nistor et alm^) grown and provided by Dan 
Leopold. Both the sapphire in the upper right corner, 
and the ion- mill damaged thin area to the lower left, are 
noise dominated in the chosen frequence window. The 
rest of the image, however, is quite uniform in color save 
for the dark near-horizontal lines across the specimen. 
These represent artificial "branch cuts" in the phase val- 
ues which gave rise (unnecessarily) to singularities during 
the gradient calculation. The magnitude of the strain in 
this GaN (003) layer is at the percent level (i.e. measur- 
able in picometers for each 0.27 nm period of traverse in 
the image), and the actual value of lattice strain in the 
growth direction is probably less given that a shift in the 
reference periodicity would likely reduce the coloration 
even further. 

The same starting image can also be used to examine 
strain components in other periodicities. For example, 
strains in the GaN (110) direction (parallel to the in- 
terface in this projection) are comparable but also a bit 
noisier, because the microscope transfer function deliver's 
the 0.17 nm periodicity less efficiently than the 0.27 nm 
periodicity in the growth direction. 

The strategy described here, sensitive to "sub-pixel dis- 
placements" like the one recommended by Seitz et alm^, 
thus arises naturally in the context of a comprehensive 
darkfield analysis. Problems with overlapping periodic- 
ities, noise, and microscope instability nonetheless con- 
tinue to limit our inferences. Hence these applications 
also provide a context for new harmonic analysis strate- 
gies that will "divide up reciprocal space" in ways which 
are more informed, to the structures being examined, 
than the "sharp-edged window method" applied here. 
We know in advance that none of the features sought in 
microscope images are likely to have sharp-edged window 
footprints in frequency space. 

VI. CONCLUSIONS AND CHALLENGES 

This paper is designed primarily for two audiences. 
For those involved in mathematical harmonic analysis. 



it is designed to illustrate a few applications of interest 
in microscopy on the nanoscale, for which optical sinc- 
windowlet analyses are in a relatively advanced stage, 
but for which digital solutions are in their infancy. It 
is also designed to highlight the windowlet interpreta- 
tion of intermediate direct and frequency space decom- 
positions. In this interpretation, all "energy conserving" 
decompositions are in a physical sense dividing up the 
frequency space "wavefield" of the orginal dataset, thus 
providing the analyst with "darkfield images" chosen to 
highlight the image patterns of interest. For those in- 
volved in microscopy, the paper is designed to illustrate 
digital versions of familar optical darkfield analysis tech- 
niques which, as in electron optic applications in princi- 
ple, use "sharp-edged" apertures. 

Challenges here posed for harmonic analysis include 
(i) the application of existing frequency-selective decom- 
positions (e.g. ridgelets and brushlets) to the study of 
nanocrystals as well as nanocrystal edges and other de- 
fects, (ii) the possible use of radon, edgelet, and other 
decompositions for the recognition of single- walled struc- 
tures (e.g. graphene sheets) in a samples ranging from 
carbon composites to dust formed in the outer atmo- 
sphere of red giants, and (iii) future development of de- 
compositions optimized for taking advantage of phase 
and amplitude information in HREM images to recog- 
nize bump (single atom) lattices buried in noise, and 
perhaps even order in paracrystallinaSS or even polymer- 
ized solids. This opportunity, to drive discovery between 
active but largely independent communities of investi- 
gators, will hopefully benefit other application areas as 
well. 
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